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1  Problem  Studied 


The  main  theme  of  the  research  conducted  under  this  grant  has  been  tp  investigate  techniques 
for  recognizing  objects  that  are  partially  hidden,  or  partially  occluded.  Two  lines  of  work  were 
pursued.  The  first  works  with  edges  extracted  from  a  gray-level  image  and  the  second  works 
with  range  data.  The  underlying  concept  in  both  cases  is  to  make  extensive  use  of  a  priori 
information  about  the  objects  that  can  occur  in  a  scene.  Geometrical  data  about  the  objects  is 
assumed  to  be  available  either  as  a  by-product  of  a  CAD  process  originally  used  to  design  the 
object  or  as  the  output  of  a  training  phase.  A  more  general  view  would  characterize  the  database 
as  simply  a  world  model. 

The  initial  phase  of  this  work  concentrated  on  recognizing  “two-dimensional”  objects  at  a 
known  distance  from  the  viewer.  Two-dimensional  objects  have  a  high  degree  of  flatness — one 
of  their  dimensions  is  much  less  than  the  other  two.  This  work  was  then  generalized  to  deal  with 
objects  that  can  be  at  any  distance  from  the  viewer.  Initial  work  on  true  “three-dimensional”  I 
objects  was  also  attempted. 

The  second  line  of  research  works  with  range  data  and  sought  to  complement  the  first 
approach  based  on  purely  gray-level  images.  The  ev<“''*"  goal  was  be  able  to  perform  three- 
dimensional  object  recognition  and  pose  (position  and  orientation)  estimation  by  using  surface  i 

I 

matching.  The  objects  in  a  scene  may  be  partially  hidden  by  one  another.  The  range  data  is 
obtained  by  laser  scanner  and  also  synthetically  using  GEOMOD  (a  solid  modeling  system).  As 
with  the  first  line  of  research  a  database  of  geometrical  information  about  the  objects  is  assumed. 
This  takes  the  form  of  a  set  of  solid  models. 

Finally,  this  research  also  addressed  the  issue  of  computation  time.  The  role  of  parallel  i 

j  i 

processing,  in  particular,  that  of  hyfcrcube  multiprocessors,  was  investigated  as  a  means  to 
reduce  the  computation  times  for  all  of  the  recognition  algorithms. 


2  Summary  of  Results 


An  early  result  in  the  work  on  object  recognition  using  gray-level  i  nages  was  to  develop  a  tech¬ 
nique  for  using  the  information  in  the  database  to  determine  which  features  (in  our  experiments 
features  were  simply  distinct  shaped  edge  segments)  were  the  most  distinguishing,  given  the  set 
of  objects  that  could  occur  in  a  scene.  Distinguishing,  or,  as  we  term  them,  salient,  features 
are  determined  during  an  off-line  training  phase  that  makes  extensive  use  of  the  database.  An 
interesting  aspect  of  the  notion  of  the  saliency  of  a  feature  is  that  it  is  context  dependent,  i.e.,  it 
depends  on  the  set  of  objects.  A  feature  that  is  highly  salient  in  the  context  of  one  set  of  objects 
may  not  be  in  another  set.  The  concept  of  saliency  allowed  us  to  achieve  improved  recognition 
of  partially  hidden  objects.  Further  details  of  the  algorithms  and  experimental  results  can  be 
found  m  the  Ph.D.  thesis  of  Jerry  L.  Turney  entitled.  Recognition  of  Partially  Occluded  Parts 
(Univ.  Michigan  1986),  and  in  the  following  papers:  “Recognizing  Partially  Hidden  Objects,” 
Turney,  Mudge  and  Volz,  published  in  the  Proceedings  of  the  IEEE  International  Conference 
on  Robotics  (1985);  “Solving  the  Bin  of  Parts  Problem,”  Turney,  Mudge  and  Volz,  published 
in  the  Proceedings  of  Vision  ’86;  and  “Automatic  Generation  of  Recognition  Features  for  Com¬ 
puter  Vision,”  Mudge,  Turney  and  Volz,  published  in  Robotica  (1987).  All  of  the  papers  have 
previously  been  transmitted  to  ARO  (see  section  on  publications). 

The  more  recent  work  using  gray-level  images  was  built  upon  “critical  point  neighborhoods” 
(CPNs).  These  are  vectors  of  edge  points  from  high  curvature  sections  of  the  contour  of  a  part  or 
object.  The  dimensionality  of  the  CPN  vectors  is  reduced  using  a  Karhunen-Loeve  transform — 
in  our  experiments  8:1  reductions  were  typical.  Tbe  model  CPNs  are  stored  in  a  database 
organized  as  a  k-d  tree.  This  allows  rapid  matching  of  the  model  CPNs  with  candidates  from 
images '  inhere  recognition  is  being  attempted.  The  use  of  k-d  trees  reduces  recognition  time  from 
a  linear  search  to  a  logarithmic  one.  A  demonstration  is  presently  working  on  2-dimensional 
jigsaw  |)uzzle  parts.  A  large  number  of  other  experiments  have  been  conducted  including  one 
proposed  by  the  Airforce  as  a  standard  benchmark.  It  is  made  up  of  parts  from  a  microswitch. 
The  details  of  this  work  are  described  in  the  paper  “Two-Dimensional  Partially  Visible  Object 
Recognition  Using  Efficient  Multidimensional  Range  Queries,”  Gottschalk,  Turney  and  Mudge, 


published  in  the  Proceedings  of  the  IEEE  International  Conference  on  Robotics  (1987).  This 
paper  has  previously  been  transmitted  to  ARO  (see  section  on  publications). 

The  lasiest  step  in  this  work  was  to  develop  a  technique  for  dealing  with  the  situation  where 
the  viewer  is  at  an  unknown  distance  from  the  objects  to  be  identified.  The  last  phase  in  this 
work  will  be  to  deal  with  truly  “three-dimensional”  objects,  i.e.,  objects  that  are  not  particularly 
flat,  and  may  be  capable  of  self-occlusion — the  handle  on  a  cup  can  be  occluded  by  the  rest  of 
the  cup  from  some  views.  Preliminary  thoughts  on  how  this  last  step  may  be  accomplished  are 
sketched  in  the  paper,  “Determining  the  Pose  of  an  Object”,  Dolezal,  Mudge,  Turney  and  Volz 
(1985).  This  paper  has  been  transmitted  to  ARO  (see  section  on  publications).  This  work  is 
ongoing. 

The  work  with  range  data  was  aimed  at  the  recognition  and  pose  estimation  of  three- 
dimensional  industrial  parts.  Three  major  subproblems  were  addressed:  1)  segmentation  and 
surface  parameter  extraction  from  a  range  image,  2)  grouping  regions  into  a  set  (a  Convex 
Region  Set),  and  3)  Convex  Region  Set  matching. 

For  segmentation,  we  restricted  our  algorithms  to  work  with  a  small  set  of  surface  shapes 
such  as  planes,  cylinders,  and  spheres,  since  such  shapes  are  common  in  industrial  parts  and  it  is 
only  necessary  to  recognize  and  locate  a  sufficient  set  of  surface  regions  to  uniquely  distinguish 
the  part  being  observed  from  all  others  that  might  be  present.  Each  region  was  extracted  using 
surface  parameters  and  surface  normals.  Segmentation  tests  were  performed  using  several  noisy 
synthetic  range  images.  This  work  was  presented  in  the  paper,  “Range  Image  Segmentation 
and  Surface  Parameter  Extraction  for  3-D  Object  Recognition  of  Industrial  Parts,”  Han,  Volz 
and  Mudge,  published  in  Proceedings  of  the  IEEE  International  Conference  on  Robotics  (1987). 
This  paper  was  transmitted  to  ARO  (see  section  on  publications). 

A  novel  method  of  grouping  segmented  regions  such  that  each  group  of  regions  represents 
a  meaningful  part  of  an  object  was  developed.  The  set  of  regions,  defined  as  a  Convex  Region 
Set(CRS),  is  obtained  by  analyzing  the  boundary  types  between  pair  of  regions.  The  boundary 
types  are  classified  as  convex,  concave,  and  jump  boundaries.  If  two  regions  share  a  convex 
boundary  it  is  assumed  that  they  are  inseparable  regions,  thus  describing  the  same  part  (object). 
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The  CRSs  are  determined  by  a  Region  Boundary  GrapKRBG)  which  is  defined  as  a  graph  whose 
nodes  represent  regions,  and  the  edges  represent  boundaries,  convex,  and  concave.  Interest'ngly, 
the  determination  of  boundary  types  does  not  require  accurate  location  of  the  boundary,  but  is 
obtained  from  the  region  adjacent  to  the  boundary.  The  technique  was  tested  using  simulated 
noisy  range  images.  Results  were  presented  in  “Region  Grouping  from  a  Range  Image,  Han 
and  Volz,  to  be  published  in  the  Proceedings  of  the  Conference  on  Computer  Vision  and  Pattern 
Recognition  (1988). 

A  technique  for  matching  sets  of  regions  (CRS)  was  developed  that  used  region  properties 
and  binary  relationships  between  regions.  The  region  property  and  binary  relationships  represent 
invariant  properties  under  three-dimensional  motion. 

Finally,  a  local  classification  of  surface  types  using  eigenvalues  of  quadric  coefficients  was 
examined.  Experimental  results  showed  that  the  quadric  invariants  are  poor  features  in  practical 
three-dimensional  object  recognition.  ' 

The  work  to  develop  parallel  versions  of  our  recognition  algorithms  made  extensive  iuse 

I  f 

of  a  64  processor  NCUBE  hypercube  multiprocessor.  The  low-level  processing  required  in 
the  segmentation  and  surface  parameter  extraction  was  successfully  ported  to  the  hypercube.  A 
speedup  of  about  30: 1  over  a  VAX  1 1/780  implementation  was  achieved.  This  work  is  reported  on 
in  further  detail  in  “Vision  Algorithms  for  Hypercube  Machines,”  Mudge  and  Abdel-Rahman, 
published  in  the  Journal  of  Parallel  and  Distributed  Computing  (1987),  and  “An  Analysis  of 
Hypercube  Architectures  for  Image  Pattern  Recognition  Algorithms,”  Mudge,  published  in  the 
Proceedings  of  the  SPIE  Conference  (1987).  Both  papers  have  been  transmitted  to  ARO  (see 
section  on  Publications).  Ongoing  work  is  examing  the  port  of  other  parts  of  the  recognition 
algorithms. 
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